Boosting Graph Neural Networks via Adaptive Knowledge Distillation

Graph neural networks (GNNs) have shown remarkable performance on diverse graph mining tasks. While sharing the same message passing framework, our study shows that different GNNs learn distinct knowledge from graph. This implies potential improvement by distilling complementary multiple models. However, distillation (KD) transfers high-capacity teachers to a lightweight student, which deviates scenario: are often shallow. To transfer effectively, we need tackle two challenges: how compact student with capacity; and, exploit GNN's own learning ability. In this paper, propose novel adaptive KD called BGNN, sequentially into GNN. We also introduce an temperature module and weight boosting module. These modules guide appropriate for effective learning. Extensive experiments demonstrated effectiveness of BGNN. particular, achieve up 3.05% node classification 6.35% over vanilla GNNs.